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- The MAIUNG DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

• Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, hovwver, may a reply be timely filed 
after SIX (6) MONTHS from ttie mailing date of ttits communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

• If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

• Failure to reply within the set or extended period for reply v\rill, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)13 Responsive to communication (s) filed on 14 May 2003 . 
2a)n This action is FINAL. 2b)IS This action is non-final. 

3) n Since this application is in condition for allowance except for foitnal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) S Clalm(s) 1-22 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) K Claim(s) 1-22 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 27 June 2000 is/are: a)n accepted or b)l3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) 0 The proposed drawing correction filed on is: a)n approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 11 9 and 1 20 

13) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-{d) or (f). 

a)nAII b)n Some*c)n None of: 

1 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 

Attachment(s) 

1 ) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). . 



2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-152) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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DETAILED ACTION 



Drawings 

This application has been filed with informal drawings which are acceptable for 
examination purposes only. Formal drawings will be required when the application is 
allowed. 



The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such 
that the subject matter as a whole would have been obvious at the time the invention was made to a person having 
ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in 
which the invention was made. 

Claims 1-22 are rejected under 35 U. S. C. 103 (a) as being unpatentable over 
U.S. Patent # 5,767.842 ("Korth") in view of U.S. Patent # 6,407,679 ("Evans et al."). 

As for claims 1, 8-10, 14, 17, and 21, Korth teaches a camera Fig.1 (2) 
corresponding to one or several cameras, a computer side of the interface there need 
only be a way to detect the fingertip motions of the keyboard operator (column 4, lines 
27-29) corresponding to one or more memories with CPU connected to the cameras, 
and processes running in the CPU that associates gesture movements with typing and 
window may appear only during the input of data (column 4, lines 54-55) corresponding 
to the produce gesture associated textual output. A virtual keyboard 3 serves as data or 



Claim Rejections - 35 USC § 103 
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command input device. From the computer side of the interface there need only be a 
way to detect the fingertip motions of the keyboard operator (column 4, lines 25-29) 
corresponding to sensing the typing gestures. It is inherent in the art that the gestures 
are associated with the most probable keys that would be typed if the keyboard were 
presented. Accordingly, Korth teaches all the claimed limitations as recited in claims 1 , 
8-10, 14, 17, and 21 with the exception of providing a classify groups of gestures 
associate each of the classes with one of the keys in the virtual keyboard. 

However, Evans et al. teach (Fig. 1 ), where each key, or finger, would represent 
several different alphanumeric characters (column 2, lines 40 and 41) and a glove 
usable in a virtual environment where each finger has a group of letters associate with 
one of the keys in the virtual keyboard (column 8, lines 13-30) corresponding to classify 
each gesture into one of the plurality classes depending on the movement of the 
gestures and associate each of the classes with one of the keys in the virtual keyboard. 
It means that each key represents a classification of several alphanumeric characters. 

It would have been obvious to utilize the finger labeled as taught by Evans et al. 
in the user command system disclosed by Korth because this would allow the user 
command to match each key sequence with one or more words in the dictionary. 

As for claims 2, 1 1 , and 1 8-1 9, Korth teaches a keyboard that should provide the 
user with the key information and feedback. Tactile feedback of virtual keyboard is given 
by the contact of the finger-tip with the working plate (column 7, lines 14-17) 
corresponding to a feedback is provided to the user on what kind of keys are associated 
with the user's gestures. 
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As for claims 3 and 12, Korth teaches a visualization of the actual key function 
allows the handling of keyboards with multiple shift keys and subgroup display for large 
characters sets (column 3, lines 1-4) corresponding to displaying keys on the display, a 
keyboard for musical instmments (column 3, line 1 1 ) corresponding to playing sound 
labels for keys and playing special sound indicators, a video sensor that is used for 
monitoring the keyboard to generate image data representing positions of an operator's 
hand with respect with the keys (column 3, lines 55-58) corresponding to displaying 
image indicators on the display, a camera Fig. 1 (2) corresponding to projecting the 
keyboard to any surface and displaying picture of the keyboard with the user's hand. 

As for claim 4, Korth teaches FIG. 2 shows the area viewed by camera 2. In the 
observed region 4 the user's hands are detected. The enlarged section 5 of FIG. 2 
represents an area of the camera's image which is chosen for further image processing 
corresponding to one or more cameras, another image evaluation procedure then helps 
the image processor to match the virtual keyboard with the template. The place-mat 
may be applied on a piece of soft tissue (e.g. polyurethane foam) to reduce the strain on 
the finger-tips from the impact on a rigid desk plate (column 7, lines 47-52) 
corresponding to one or more memories with CPU connected to the cameras, and A set 
of simple intuitive 'gestures' may be defined to control the operation of a virtual 
keyboard (column 6, lines 7 and 8) corresponding to processes running in the CPU that 
associates gesture movements with typing and produce gesture associated with textual 
output and a gesture capturing module that captures gestures through camera sensors, 
and the shape of the hands can be used to send gestures with additional information to 
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the optical interface (column 2, lines 55 and 56) corresponding to an integrator module 
that integrate sequence of candidates of most probable keys into unique output key 
sequence. 

As for claim 5 and 6, Korth teaches a method of user command or data 
acquisition in a processing system that allows for analysis of human hand and finger 
motions. The activities are then interpreted as operations on a physically non-existent 
input device. The resulting optical user/computer interface complements the visual 
screen-to-user path with a hand-to-image data acquisition link from the user to the 
processor unit. It allows for the emulation of different types of manually operated input 
devices (mouse, keyboard, etc.). Mechanical input units can be replaced by virtual 
devices, optimized for the current application and for the user's physiology. The speed, 
simplicity and avoidance of ambiguity of manual data input will be maintained. Sensing 
of finger motions can generate feedback to reduce the hazard of repetitive strain injury 
(RSI) syndrome (abstract). 

As for claim 7, Korth teaches a method of data input using video sensors for 
monitoring positions of an operator's hand with respect to keys on a virtual keyboard 
optically produced on a surface. 

As for claims 13, and 20, Korth teaches a keyboard needs not to exist physically, 
a virtual keyboard 3 serves as data or command input device. From the computer side 
of the interface there need only be a way to detect the fingertip motions of the keyboard 
operator. The task of the keyboard is to provide information about the available keys 
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and their location, to sense the depression of the key by the operator and to produce 
some feedback when a key has been touched (column 4, lines 21-29). 

As for claims 15 and 22, Korth teaches a set of simple intuitive 'gestures' may be 
defined to control the operation of a virtual keyboard (column 6, lines 7 and 8) 
corresponding to means for associating gesture classes with individual typing keys. 

As for claim 16, Evans et al. teach the process loops back to select another 
word. This process continues until all the words have been placed in the data structure 
(column 4, lines 62-64) corresponding to providing training data in words or sentences 
with certain timing data. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jean Lesperance whose telephone number is (703) 
308-6413. The examiner can normally be reached on from Monday to Friday between 
8:00AI\/I and 4:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richard Hjerpe, can be reached on (703) 305-4709 . 

Any response to this action should be mailed to: 
Commissioner of Patents and Trademarks Washington, D.C. 20231 

or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 



Conclusion 
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Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal drive, 
Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the technology Center 2600 Customer Service Office 
whose telephone number is (703) 306-0377. /) 
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